When do libraries of articulated images obey an isometric embedding?

in image understanding and image coding, it could be useful to ‘learn’ the structure of such image articulation manifolds and to recover the underlying parameters (location, scale, etc.) from unlabeled data. This could be important for recognizing articulated vehicles in target recognition, and for understanding articulated faces in facial recognition. The general problem of learning the shape of a manifold from scattered observations has been around for a long time; it has been the source of many multivariate techniques, including principal components analysis, independent components analysis, multidimensional scaling, selforganizing mappings, and other important methodological developments. Recently, Tenenbaum et al. [9] proposed the Isomap procedure as a general tool for recovering the unknown parametrization underlying a set of digital images, {Ii}, of faces in various attitudes and articulations. The general principle of Isomap is to measure distance between images, not using Euclidean distance (which is ignorant of the manifold structures), but using distance according to the shortest path in the nearest neighbor graph; and to use this graph distance as input to a classical “principal coordinates” multidimensional scaling procedure. 1.1 Validating Isomap Tenenbaum et al. [9] published a few interesting examples, for example mapping out the parameters underlying a face seen from a variety of viewpoints. These empirical successes lead to the ... Obvious Question: how “correct” is the Isomap procedure; does it really recover the “true” underlying parametrization of families of articulated images? This leads immediately to the ... Obvious Approach: Test Isomap for synthetic data where we know a priori “the natural” parametrizations and see if it can recover the parametrization of image manifolds. In following the ‘obvious’ approach, we would construct datasets of artificial images undergoing standard articulations – translations, rotations, etc. – and ask if the Isomap parametrization correctly discovers the underlying parametrizations. While the question and the investigative approach seem clear at first glance, on closer inspection, it is not clear that one can push this line of investigation very far. There are several reasons a purely empirical investigation based on running Isomap on artificial examples may not be enlightening. • Sampling issues. Whether Isomap works or not might depend on how many model images Ii are in the database. In some vague sense, it will be important for these images to be well-distributed across the image manifold, but exactly what this means in a specific instance is unclear. Consequently, if Isomap ‘fails’, this may be due to poor data sampling rather than any intrinsic property of Isomap. • Digitization issues. In some sense, the fact that images are discretized into pixels makes them ‘noisy’ / ‘blocky’; this again causes some difficulty in interpreting ‘failure’ of Isomap – is it due to the pixelization or is it intrinsic to the type of articulation? • ‘Big Picture’ issues. Empirical work really doesn’t give us an intellectual framework that we can leverage into other settings, at least not the kind of framework that might be possible by a more theoretical approach. For these and other reasons, we propose to develop an alternate framework for understanding Isomap. We think of an image as a function I(x) of a continuous variable x ∈ R. We consider articulations of a base image I0, producing a family of images {Iθ : θ ∈ Θ}, where θ is the parameter of the articulation and Θ is the parameter space. Examples we will consider include translation families, where Θ = R, Iθ(x) = I0(x − θ). In this “continuum” viewpoint, neither sampling nor digitization can cause problems, and a clear intellectual framework exists naturally.